Markov processes conditioned on their location at large exponential times

Nonequilibrium Markov Processes Conditioned on Large Deviations

Accelerated decomposition techniques for large discounted Markov decision processes

Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...

Robustness of Markov processes on large networks

A metric on the space of probability measures on the state of a large network is introduced, with respect to which the stationary measure of a Markov process on the network is proved under suitable hypotheses to vary uniformly smoothly with parameters and the rate of relaxation to equilibrium to never suddenly decrease.

Markov Decision Processes with Constrained Stopping Times

The optimization problem for a stopped Markov decision process is considered to be taken over stopping times constrained so that E 5 for some xed > 0. We introduce the concept of a randomized stationary stopping time which is a mixed extension of the entry time of a stopping region and prove the existence of an optimal constrained pair of stationary policy and stopping time by utilizing a Lagra...

ON THE INFINITE ORDER MARKOV PROCESSES

The notion of infinite order Markov process is introduced and the Markov property of the flow of information is established.